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Retention Research: Issues in Comparative Analysis 



Abstract 

Conducting institutional research on a complex issue such as retention provides the opportunity to utilize 
the method of comparative analysis whereby numerous independent comparisons can provide the 
institution with a more complete picture of an issue. We have done numerous types of comparative 
analyses on retention at our institution over the last several years. Each of these comparisons provides a 
unique perspective on the retention and success of students at our institution, along with its own unique 
challenges and problems. In presenting the results of the various comparisons, we identify and discuss 
these perspectives and challenges as they have limited the effectiveness of our methodology. 



Introduction 

Conducting institutional research on a complex University issue such as retention provides 
unique challenges to the institutional researcher. The limitation of influencing action in retention is that 
people make judgments about the need to improve retention based on a simple rule: “How are we 
doing?” is the prompt and if the answer is “We do as well as others like us” or “As well as we can 
expect to do,” then it is not likely that things will change. To bring data to bear on these questions and 
to further look for the specific areas where an institution might want to focus its resources, a multi- 
method approach seems prudent. We have done numerous types of comparative analyses on retention in 
our institution over the last several years. This paper discusses those comparative analyses and the 
issues involved in the utilization of the information. The first part in this discussion provides a brief 
background on retention as a strategic issue. The second part looks at the types of comparative studies 
that can be done with examples from what we have done. The third part looks at some of the things we 
have learned within the template of Chickering and Gamson’s (1987) seven principles good practice in 
undergraduate education. 

Background on Retention 

Two comprehensive conceptual models have emerged to guide the study of student persistence: 
Tinto’s Student Integration Model (Tinto, 1975, 1993) and Bean’s Student Attrition Model (Bean, 1980. 
1983, 1990). Tinto (1975) advanced the idea that the fit between the student and the institution plays a 
key role in the likelihood of persistence (Cabrera, Nora and Castaneda, 1993). Accordingly, the degree 
of institutional commitment a student feels, and the subsequent persistence, is shaped by the congruence 
between student motivation and ability and the institution’s academic and social characteristics. 

Bean (1983) reflected on the link between attitudes and behaviors as theorized by Fishbein and 
Ajzen (1975), who wrote that through a person’s intentions attitudes predict subsequent behaviors. 
Bean’s Student Attrition Model posits that students’ beliefs about their experiences in school affect their 
intention to stay and subsequent persistence (Bean, 1990). This model also recognizes the influence of 
external (to the institution) factors on retention, something many researchers have found missing from 
Tinto’s model (Bean, 1985; Cabrera, Nora and Castaneda, 1993). 

These two theories have spawned an enormous amount of research on the area of student 
retention. Many of these studies were done to confirm or deny the basic tenets of the theories (Cabrera, 
Castaneda, Nora and Hengstler, 1992; Nora and Cabrera, 1993; Pascarella and Chapman, 1983; 
Pascarella and Terenzini, 1983; Stage, 1989) or have integrated these two approaches into a singular 
framework (Cabrera, Nora and Castaneda, 1993). Other studies have looked at different strata within 
the student population, including: 
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• Minority students (Bean and Hull, 1984; Fuertes and Sedlacek, 1994; Grandy, 1998; King 
and Chepyator-Thomson, 1996; Nora, 1987; Ting, 2000); 

• Commuter students (Johnson, 1997); 

• Graduate students (Cooke, Sims and Peyrefitte, 1995; King and Chepyator-Thomson, 1996); 

• Two-year college students (Bers and Smith, 1989; Pascarella, Smart and Ethington, 1986); 

• Transfer students (McCormick and Carroll, 1997); 

• Non-traditional and adult students (Bean and Metzner, 1985; Shields, 1994). 

Still others have looked at a multitude of factors that potentially affect persistence, including: 

• Academic aptitude (Kennedy, Gordon and Gordon, 1995; McGrath and Braunstein, 1997; 
Fuertes and Sedlacek, 1994); 

• Student-Faculty interactions (Nagda et al., 1998; Pascarella, 1980); 

• Student services (Junn, Fuller and Derrell, 1996; St. John, 2000; Turner and Berry, 2000); 

• Financial factors (Cabrera, Nora and Castaneda, 1992; Cabrera, Stampen, and Hansen, 1990; 
McGrath and Braunstein, 1997; Nora and Horvath, 1989); 

• Learning communities in and outside of the classroom (Baker and Pomerantz, 2000; Johnson, 
2000; Tinto, 1997). 

Retention as a Strategic Issue 

If so much is known about student retention, why is reluctance to make changes aimed at 
improving retention? One reason could be that the changes require shifts in administrative and faculty 
cultures. These shifts must be learned, and for learning to occur, a strategic process is necessary to 
relate the shifts to the context of the institution and the values of the major audience. Delaney (1997) 
argues that for institutional research to be successful, it needs to go beyond simply collecting and 
reporting data. The major obstacle to supporting efforts to better understand student retention is 
persuading stakeholders at all levels of the institution of the importance of student retention. As 
McLaughlin, Brozovsky and McLaughlin (1998) put it, institutional researchers must persuade 
institutional stakeholders that student retention should be treated as a strategic issue, which they define 
as issues that can have serious consequences for the long-term success of the institution. 

That retention is an important issue seems beyond debate, given that a variety of federal, state 
and private consortia request the reporting of these data. Also retention data are being used as indicators 
of academic quality in the computation of institutional scores for the U.S. News and World Report 
annual college rankings (Graham and Morse, 1998). While the debate continues over the efficacy of the 
U.S. News methodology (Cronin, 2000), few can argue the importance of such rankings in the 
enrollment management and institutional research process (Monks and Ehrenberg, 1999). Yet, getting 
this issue onto the institutional radar so that key organizational stakeholders take notice appears to be a 
near-universal challenge for institutional researchers (McLaughlin et al., 1998) 

The complexity involved in creating any systematic evaluation of a key strategic issue is 
demonstrated by Dolence (1991). To develop an effective research agenda, he suggests an evaluation 
matrix consisting of the relationship between a dozen criteria and four specific dimensions: how 
decisions are made; what information and data are available; what planning processes the institution 
engages in; and if and how the “right” people in the institution are involved. The twelve criteria he 
suggests include leadership, comprehensiveness, key performance indicators and participation. Table 1 
displays four of Dolence’s criteria arrayed across the four dimensions considered. The more of these 
questions that are answered in the affirmative, the more likely it is that the institution is primed for 
utilization of the information obtained through the study. 
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Table 1 

Sample Evaluation Matrix for Retention Studies 
(adapted from Dolence, 1991) 



Criteria 


Decisions 


Information and Data 


Processes 


People 


Leadership 


Are decisions made 
with an awareness of 
environmental trends 
and institutional 
strengths, weaknesses, 
values and resources? 


Are the data collected 
and analyzed 
consistent with the 
decisions faced by the 
institution? 


Does the institution 
engage in a formal 
planning process that 
sets the key decisions? 


Are the communication 
processes of the 
institution adequate to 
sustain a strategic 
focus? 


Comprehensiveness 


Are individual and unit 
responsibilities clearly 
articulated and 
understood? 


Is the quantity of data 
analyzed sufficient for 
sound decisions? 


Are processes 
reviewed and revised 
to eliminate 
redundancy? 


Are the right people 
involved in the 
retention of students? 


Key Performance 
Indicators 


Are there clear 
indicators of success 
with respect to 
retention and are they 
articulated and 
understood? 


Are the indicators 
measurable and are 
data available 
routinely? 


Are the indicators of 
success monitored in 
an appropriate time 
frame? 


Are the indicators of 
success tied to specific 
individuals or units? 


Participation 


Is there adequate 
participation by 
constituents in 
decision-making? 


Is information shared 
in an appropriate 
manner? 


Is there a formal 
participative process 
employed? 


Do people feel like 
they are part of the 
strategy of the 
institution? 



Indeed, information utilization is a primary concern for institutional and evaluation researchers, 
as they frequently encounter indifference from decision-makers and other institutional stakeholders 
(Ewell, 1989). Kinnick (1985) distinguished between technical and organizational obstacles that impede 
the otherwise smooth flow of information from researcher to decision-maker. Kinnick and Ricks (1993) 
provide a model (adapted from Ewell, 1989) of the factors that influence information utilization. These 
factors essentially fall into characteristics of the organization (the power structure, the organizational 
climate, and the communication structure), the researcher, the decision-maker and the information. 

So how does the researcher avoid these pitfalls and effectively convey the information to the 
decision-maker? Kinnick (1985) offers a number of organizational and data presentation strategies for 
increasing the use of outcomes information. The organizational strategies involve attempts to 
incorporate the new information into ongoing institutional activities, such as academic program review 
and strategic planning and forecasting. The data presentation strategies include more use of graphics, 
providing multiple comparisons across different institutional categories, and utilizing short, issue- 
specific reports. Kinnick and Ricks (1993) use a case study approach to demonstrate stages of 
implementing a change process. In a successful retention project one of their specific recommendations 
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is that “the use of multiple strategies for gathering information to address issues of retention is 
imperative" (p. 68). 

Levitz, Noel and Richter (1999) suggest presenting the issue of student attrition in terms of lost 
tuition revenue. By creating a revenue model that calculates tuition dollars lost through student attrition, 
they found that even a small increase in student retention resulted in significant savings. Since retention 
is ultimately a long-term goal and will result in institutional change, Levitz, Noel and Richter also 
suggest tips on how to get key stakeholders involved with what will most likely be viewed as an 
unwanted or unneeded change. They suggest carefully selecting a new retention task force to establish 
research guidelines and set reasonable priorities and action plans. They also state that it is important to 
increase the institutional understanding of what retention means, and to use research to challenge the 
myths and preconceptions of key administrators and faculty members. 

McLaughlin et al. ( 1 998) proposed a sequence of events that could be utilized to change 
institutional attitudes about the importance placed on retention efforts. They suggest that institutional 
researchers must challenge the long held preconceptions and complex predispositions of key 
institutional stakeholders. By drawing an analogy to the work of Kubler-Ross (1993) on coping with 
death, they suggest that stakeholders go through several stages before they are willing to accept what is 
perceived as undesirable change. These stages are, in order of occurrence, denial, hostility, bargaining, 
depression and finally acceptance. They recommend a multimethod approach for research in retention 
in order to move the institutional stakeholders through these stages to acceptance. 

Seven Principles of Good Practice in Undergraduate Education 

The framework for considering the value of the various analyses in the context of the institution 
comes from the Chickering and Gamson (1987) study of the seven principles of good practice in 
undergraduate learning. These seven principles are: 

1 . Good practice encourages contact between students and faculty 

2. Good practice develops reciprocity and cooperation among students 

3. Good practice uses active learning techniques 

4. Good practice gives prompt feedback 

5. Good practice emphasizes time on task . 

6. Good practice communicated high expectations 

7. Good practice respects diverse talents and ways of learning 

Evidently, these principles are directed at faculty and university administrators and the students are the 
beneficiaries. By re-framing these principles to reflect institutional researcher and stakeholder 
interactions, we feel that we have an interesting framework in which to evaluate the efficacy of our 
communications with the institution regarding studies of student retention. In this case, the principles 
would be directed at institutional researchers with various audiences of stakeholders being the 
beneficiaries. 

First the institutional researcher must determine who the audience is for retention research. This 
looks at the “people” column of Dolence’s matrix. At DePaul, as is probably the case at many 
institutions across the country, there are numerous audiences. Some of these are not specific 
individuals, but general groups of stakeholders like students, faculty and enrollment management 
administrators. For example, the student is interested in his/her likelihood of success at your 
institutions, compared to other institutions. The faculty - as the public face of the institution for the 
student - have a general interest as well, as they provide the primary mechanism by which students 
persist and graduate. Therefore, they are usually the ones asked to change and to learn new ways. 

Offices for enrollment management are interested in both marketing the institution and seeking the niche 
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of students who will perform well at the institution. They would also like the see the number of 
continuing students increase, so they can be more selective when bringing in new students. This is also 
related to the publication of US News and other rating groups that use graduation and retention as 
measures of institutional desirability. 

However, at most institutions, there are probably two groups of individuals who would be 
considered primary audiences. As a private institution, DePaul’s resources are determined and allocated 
based on the credit hours generated by the different colleges. Thus, the individual college deans have a 
vested interest in the persistence of their students. Some public institutions, such as those in Virginia 
and South Carolina, also need to attend to their legislatures since state resources can be allocated based 
partly on student retention and graduation. These individuals would represent one primary audience for 
an institution. 

The second primary audience consists of those individuals assigned the responsibility to monitor 
and improve student success. At DePaul, three of the major groups of individuals assigned this 
responsibility are the First Year Program, University Academic Advising, and the Office of 
Multicultural Student Affairs. Since the majority of losses to new freshman cohorts occur between the 
first and second years, the administrators and faculty responsible for the First-Year program constitute a 
major audience of our research on this group of students. Many transfer students have reported 
problems and difficulties in the transfer advising process, an issue of great concern to the administrators 
of the new Advising Center. This center has become a key audience to studies on transfer student 
retention, as well as student success in general. The Office of Multicultural Student Affairs runs a 
“voluntary, comprehensive, first-year retention initiative” designed to assist first-year/transfer 
multicultural students in their transition to our university and is a key audience to studies focusing on the 
differences in retention between different student demographic groups. 

The Present Study 

In an effort to advance the institutional conversations on the retention and graduation of students 
at DePaul University, the Office of Institutional Planning and Research has undertaken a number of 
independent studies on this issue. The following are some of the specific comparisons we have done: 

1 . Internal Comparisons: This is the method of retention studies most commonly utilized by 
institutions and typically one of two methodologies, if not both: 

> Cohort-by-cohort comparison - where a set of cohorts are identified and their one-year 
retention and six-year graduation rates are compared. 

> Longitudinal Comparisons - where cohorts are tracked over multiple years of enrollment, 
allowing for the researcher to look at the time and magnitude of the losses to the cohort. For 
example, while the majority of losses to the cohort occur after the first-year, an equal 
percentage is lost between the second and sixth years, although the reasons for the losses may 
change. 

These internal comparisons allow for discussions of improvements and also possible interpretations 
concerning the impact of various programs and changing contextual conditions. 

2. External Comparison: Being able to benchmark your institution's retention with that of other 
institutions provides an external reference of how well you are doing. This can be a powerful tool 
for pressing upon stakeholders the importance of retention as a strategic issue. Using an approach 
similar to the one outlined by Camp (1989), we identified a list of institutions that are similar in key 
statistical characteristics. Also we selected these from the other institutions who belong to the 
Consortium on Student Retention Data Exchange (CSRDE) who provide comparison data for a set 
of selected institutions. 



3. Entry/Intent Comparisons: The typical retention study focuses on the cohort of first-time, full-time 
degree seeking students. However, these students are quickly becoming the minority at our 
institution as well as in many other institutions. Therefore, we have done studies on the retention of 
transfer students, professional students, and part-time adult students. The studies for each of these 
different student populations provide a more complete institutional picture of student retention than 
can be understood from strictly looking at the traditional student. 

4. Demographic Comparisons: This is not actually a separate class of studies, as demographic 
comparisons are conducted as a routine part of all studies on retention. However, it is important to 
compare the retention of students based on their demographic character, including race, gender, 
college and program/major. These are essential for demonstrating to departmental and college 
leaders that retention is a key strategic issue at their organizational level and involve the comparison 
of retention rates of colleges and the graduation rates at the program level. 



Research and Results 
Internal Comparisons 

In this methodology, a cohort, or set of cohorts, is identified and compared or tracked over multiple 
years, allowing for the researcher to look at the time and magnitude of the losses to the cohort. Through 
a comparison of retention and graduation rates across multiple cohorts, a researcher can demonstrate the 
effectiveness of programs initiated to improve retention. By studying one cohort over several years, the 
researcher can demonstrate when students typically stop matriculating. 

Cohort-to-Cohort First-Year Retention and Sixth-Year Graduation Rates 

Overall, DePaul’s average 
first-year retention rate for 
the 1989 through 1998 has 
fluctuated between 75% 
and 85%, showing a slight 
increasing trend over the 
last few years. Where 
available, corresponding 
graduation rates have 
shown a similar pattern of 
change, where increases 
from one cohort to the next 
in retention correspond 
with similar increases in 
subsequent graduation for 
the cohort. Previous 
research suggests that 
student support in the form 
of orientation, advising and 
introductory courses can lead to higher retention rates by easing some of the common anxieties and 
concerns that first-year students often have (Levtiz, Noel & Richter, 1999). The overall retention rate 
has been generally increasing since the 1995 cohort, the year in which DePaul’s revised First-Year 
Program was fully implemented, including many programs for new freshman designed to increase their 
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academic and social integration. It is hoped that these increases in retention will result in similar 
increases in graduation. 

Similar charts were prepared for gender and race comparisons. These charts showed that 
females at DePaul have an average retention rate (81.1%) similar to males (80.6%), and both groups 
have maintained similar retention rates for most cohorts, though rates for females were usually a few 
percentage points above males. African-American students at DePaul (77.3%) had the lowest overall 
average retention rates. However, the retention rate for African-American students at DePaul has also 
shown the largest overall increase of any ethnic group - 15 percentage points since 1989. Asian 
students have the highest overall retention (85.8%) of any ethnic group at DePaul, and have shown rapid 
increases between the 1994 and 1998 cohorts. The only other group to show this kind of spike in 
retention is Hispanic students, with an increase of 10 percentage points over three cohorts and an overall 
retention rate of 82.7%. In general, retention rates for minority students at DePaul is increasing faster 
than Caucasian students, and in the 1996 and 1998 cohorts, minority student retention rates surpassed 
those of Caucasian students. 

We also conducted retention analyses for each college/school at DePaul, as well as on some 
different academic indicators. DePaul has eight colleges, but freshmen retention is only studied in six 
of them (the Law School and the School for New Learning (an Adult Education school) were excluded 
for obvious reasons). Our Business school had the highest overall average retention rate of any college 
at DePaul (83.7%) while Liberal Arts and Sciences has had the lowest average retention rate of the three 
high-enrollment colleges (80.1%). For our Computer Science school, retention rates in recent cohorts are 
much more stable than previous cohorts due in part to rapidly increasing enrollments. 

A student’s first-year GPA is the cumulative GPA for all courses taken prior to the Fall term of 
the second year. Students earning a first-year GPA of less than 2.0 have extremely low retention rates, 
with an overall average of 47.5%, over 30% lower than any other GPA group. These low retention rates 
are mostly due to the academic probation policies, which state that students earning below a 2.0 GPA for 
any academic term have two academic terms to improve their GPA to at least 2.0 or the student is 
dismissed. Therefore, most students with a first-year GPA below 2.0 cannot be classified as “voluntary 
withdraws”. 

The remaining four GPA 
categories have much higher 
retention rates, and while there 
were larger gaps between these 
groups in earlier cohorts, they 
have become more similar to 
each other in the recent 
cohorts. Students earning a 
first-year GPA between 2.0 and 
2.49 had an average retention 
rate of 80.3%. This rate 
increased 12 percentage points 
between the 1989 and 1998 
cohorts, and has remained 
between about 85% and about 
90% between the 1995 and 
1998 cohorts. Retention rates 
for students earning a first-year 
GPA between 2.5 and 2.99 and 
between 3.0 and 3.49 both have an overall average retention rate of 85.3%, and retention rates for both 




1989 1990 1991 1992 1993 1994 1 995 1996 1 997 1996 
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groups have remained between 80% and 90% since the 1989' cohort. The highest retention rates are 
those of students earning a first-year GPA of 3.5 and higher, with an overall average of 89.2%. First- 
year retention rates for these students have remained between 85% and 95% across all ten cohorts. 



Figure 3 



One other area of interest to 
stakeholders at DePaul is the 
performance of students who 
matriculate at DePaul as new 
freshmen after having 

completed the Bridge 

program. The DePaul 
Summer Bridge Program is 
an intensive 5-week college 
preparatory program for 
students requiring assistance 
in reading, writing, math, or 
study skills that takes place 
on campus during the 
summer term prior to cohort 
entry. Students participating 
in Bridge during the summer 
are included in the 
subsequent Fall entering 

cohort provided they enrolled as full-time freshmen in the fall. Participating students receive academic 
support throughout the first Fall term, and students who earn a 2.0 GPA or better in the Fall quarter are 
fully admitted to DePaul University. Retention rates for students participating in the Bridge Program 
have been higher than those of non-Bridge participants for six out of the ten cohort years, and have an 
average of 82.3%, compared to an average of 80.7% for non-Bridge participants. Retention rates for 
both groups are generally increasing, however, the Bridge students’ retention rates have been increasing 
at a faster rate than non-bridge students, with an overall increase of over 12 points across all ten cohorts 
compared with 4.8 points for non-Bridge participants. The inherent nature of the Bridge program is to 
provide extensive academic and social support to participating students, to better help them adapt to 
DePaul. This support is extended through the Fall quarter of the first year. Thus the higher level of 
student support for Bridge students is a key factor in explaining why retention rates for Bridge students 
would be higher than that of non-Bridge students (Tinto, 1975; Levitz, Noel and Richter, 1999) even 
though the non-Bridge students are, by definition, more academically prepared for college. 
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Figure 4 

Attrition of New Freshmen Through 6 years 




Longitudinal Studies of First-Year Retention and Sixth-Year Graduation Rates 

Through a comparison of retention and graduation rates across multiple cohorts as shown above, a 
researcher can demonstrate the effectiveness of programs initiated to improve retention. By a 
longitudinal study of one cohort, the researcher can demonstrate when students typically stop 
matriculating. As is shown, 

DePaul typically loses 
about 15-20% of new 
freshmen after the first 
year. Our sixth-year 
graduation rate tells us that 
approximately 55-60% of 
the original cohort of new 
freshmen graduates after 
six years. Therefore, 
between the second and 
sixth year, DePaul has lost 
another 20% of the original 
group of new freshmen on 
top of the 20% lost after the 
first year. In other words, 
by the end of the sixth year, 

DePaul has lost 400 of 
every 1 000 freshmen 
enrolled. These analyses 
also demonstrate the 
consistency of this matriculation pattern. 

Although similar analyses have been done looking across race, gender, college and other 
demographic variables, one study that was particularly well received looked longitudinally at Bridge 
students. In particular, we compared Bridge and non-Bridge students on the likelihood that their 
withdrawal was voluntary versus non-voluntary. By the sixth-year, twice as many Bridge students than 

non-Bridge students leave DePaul 
with a cumulative GPA below 2.0 (not 
passing). The overall average sixth- 
year attrition rate for Bridge students 
(averaged over the 1988 - 1993 
cohorts) is 47.6%, about half of those 
students had a cumulative GPA below 
2.0. Figures 5a and 5b show the 
progression of voluntary and non- 
voluntary attrition for Bridge and non- 
Bridge students (respectively). While 
the overall attrition rates of Bridge and 
non-Bridge students remain very 
similar during the first through sixth 
years, the proportion of students “Not- 
Passing” is much greater for Bridge 
students than non-Bridge students. 



-191*1 Cohort 
-1995 Cohort 



- 1992 Cohort 
- 1996 Cohort 



-1993 Cohort 
-1997 Cohort 



- 1 994 Cohort 
-1998 Cohort 



Figure Sa 

First-Through Sixth-Year Attrition Rates for Bridge Students 
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Figure 5b 

First-Through Sixth-Year Attrition Rates for Non-Bridge Participants 



Figures 5a and 5b illustrate a nearly inverse relationship between Bridge status and attrition status - 
where the percentage of Bridge students 
who were “Not-Passing” at the time of 
withdrawal is almost equal to the 
percentage of non-Bridge students who 
were “Passing,” and vice-versa. 

Although student support and services 
extended to Bridge students during 
their first year have impacted their 
retention rates, these services do not 
impact the graduation rates of these 
students. This might suggest the need 
for extending the duration of support 
offered to Bridge students, especially 
in the academic arena, in order to 
improve retention and graduation rates. 

This study, paired with the 
comparisons by GPA resulted in one 
administrator spearheading a larger 
study of what contributes to student success at DePaul. 




External Comparison: Benchmarking Student Retention and Graduation 

Benchmarking is an important tool to help institutional stakeholders, the public, the media, students and 
students’ families understand how an institution compares to its peers on key measures of student 
success. In a time when much stock is being placed on accountability, and publications such as U.S. 
News & World Report are using retention and graduation as key indicators of academic quality in their 
college rankings (Graham & Morse, 1998), it is extremely important that these numbers are understood 
in context. 

In order to provide both a broader view of retention and graduation rates as well as determine a 
point of reference for DePaul’s retention and graduation rates, DePaul’s rates were compared to data 
from eighteen reference institutions. For another study, we had generated a reference group of 60 
institutions based upon their statistical similarity to DePaul, using a method similar to that outlined by 
Camp (1989). This list was compared to the list of institutions participating in the Consortium for 
Student Retention Data Exchange (CSRDE) annual survey of retention, administered by the Center for 
Institutional Data Exchange (C-IDEA) at the University of Oklahoma. The CSRDE consists of about 
300 institutions from across the country and collects retention and graduation data by race and gender 
for several entering cohorts. Member institutions can receive data for a group of peer institutions of 
their choice for comparison purposes. Consequently we came up with a reference group consisting of 
1 8 institutions. This procedure is outlined below. 

This reference group was selected using three sources: 

1. A list of institutions used in the most recent faculty salary comparison 

2. A list of institutions from the 1 999-2000 Consortium for Student Retention Data Exchange 
(CSRDE) survey 
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3. A list of urban Doctoral I, Doctoral II and Research II institutions with enrollments between 
10,000-30,000 




1998 Reference Institution Statistics 
Cohort Information Highest Lowest 


DePaul 


Total Cohort 


3,124 


476 


1,443 


Female Cohort 


1,814 


268 


844 


Male Cohort 


1,310 


204 


599 


African-American Cohort 


551 


13 


123 


Hispanic Cohort 


658 


0 


176 


Asian Cohort 


1,038 


0 


139 


Caucasian Cohort 


1,749 


131 


904 


Minority Enrollment* 


35.0% 


2.0% 


33.3% 


Students 24 or older* 


7.0% 


0.0% 


1.8% 


Living on Campus* 


96.0% 


0.0% 


54.2% 


First-Term GPA < 2.0* 


37.0% 


6.0% 


12.6% 


Part-Time Enrollment 








(All Undergraduates) 


48.0% 


4.0% 


37.8% 


* At least one institution reported no value for this field 



Source: 1999-2000 CSRDE Report, OIPR Fact File 



These institutions represent a mix of Private, Public 
and urban institutions whose student bodies resemble 
that of DePaul on several key measures such as 
institutional control, Carnegie Classification 1 , size 2 , 
freshmen cohort size, minority student enrollment, 
average undergraduate student age, percentage of 
part-time undergraduates, percentage of 
undergraduates living on campus, and percentage of 
undergraduates earning a first-term GPA below 2.0. 
All institutions participated in the 1999-2000 CSRDE 
Retention Survey. DePaul falls near the median of 
the reference group in most of these institutional 
measures. 





Institutional Cha: 
Control 


racteristics of Reference Group and 

Carnegie Classification 1 


l DePaul 

Size 2 


# Peers 
DePaul 


Public Private 

11 7 

S 


Doctoral Doctoral Master’s I 
Extensive Intensive 

6 11 2 

✓ -• - - 


Large Medium 

6 12 
✓ 



Source: 1999-2000 CSRDE Report, Chronicle of Higher Education. 



The data for first-year retention 
and sixth-year graduation were 
plotted on a box plot, which 
illustrates the range of rates over 
the entire reference group. 



1 The Carnegie Classification codes were updated in August 2000. The classifications in this report reflect these new 
changes. 

2 The 1999-2000 CSRDE Report defines ‘Large’ as an institution with having a total enrollment of at least 18,000 and ‘Medium’ as an 
institution having a total enrollment between 5,000 and 17,999. 
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Benchmarking First-Year Retention: DePaul vs. Reference Group 

Figure 6a plots the distribution of first-year retention rates across all reference institutions for the 1998 
entering cohort against the actual first-year and sixth-year rates of DePaul’s 1998 entering cohort. Plots 
were created for the entire entering cohort, and for gender and ethnicity. An ‘X’ marks DePaul’s first- 
year retention rate for each group in Figure 6a. The number of reference institutions included in each 
breakdown appears along the bottom of the chart; note that not all institutions reported enrollments for 
each demographic group for the 1998 entering cohort. 
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Figure 6a 

DePaul Retention and Reference Group Retention: 1998 Entering Cohort 
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For the entire entering cohort of 1998, DePaul’s retention rate (83.9%) was 6.3 percentage points higher 
than the reference group median (77.6%), and ranked at the 74 th percentile of the distribution, meaning 
DePaul’s retention rate is higher than about three-quarters of the reference institutions. First-year 
retention for females at DePaul also exceeded the peer group median by about 3 points, and ranked at 
the 69' percentile. The females in the reference group had a higher retention rate than males, with a 
median about 3 points higher than males. Retention for DePaul males ranked at the 87 th percentile, 
meaning DePaul males have higher rates than females at DePaul for the 1998 cohort and also females in 
the reference group. First-year retention for DePaul males exceeds the reference group median by over 
8 percentage points. 

DePaul’s first-year retention rate for African-American students (84.6%) was higher than that of 
the reference group (77.0%), ranking at the 85 th percentile of the reference group distribution. Hispanic 
students in the reference group have the largest range of first-year retention rates (from 50.0% to 
95.4%), with a median of 75.0% - 1 1 .4 percentage points below DePaul. Compared to the distribution, 
Hispanic students at DePaul ranked at the 8 1 st percentile. Asian students in the reference group had the 
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smallest range (64.4% to 97.0%). DePaul ranked in the 79 th percentile of the reference group 
distribution for Asian student retention, with a rate of 92.1%. The median first-year retention rate for 
Caucasian students in the reference group was 77.5%, about 4 points below DePaul’s retention rate 
(81.7%), which ranked at the 70 th percentile. 

The differences between DePaul and the reference group vary greatly and while DePaul appears 
to perform better than the reference group in some areas (male and Hispanic student retention), there is 
little difference between DePaul and the reference group in other areas (female and Caucasian student 
retention). But for most groups DePaul’s retention rate for the 1998 entering cohort is at or beyond the 
75 th percentile of the reference group distribution. In general, we can conclude that DePaul’s retention 
rate is at about what we should expect, given our reference point, and in some cases, better than we 
should expect. 

Benchmarking Sixth-Year Graduation Rates: DePaul vs. Reference Institutions 

Figure 6b shows DePaul’s sixth-year graduation rate for the 1993 entering cohort was plotted against the 
distribution of graduation rates among the eighteen reference group institutions. There were only two 
years of available reference data for graduation rates, and these rates did not differ greatly between the 
1992 and 1993 cohorts for any institution. For this reason, the average between the graduation rate for 
the 1 992 and 1 993 cohorts was taken for each institution. The number of reference institutions included 
in each breakdown appears along the bottom of the chart; note that not all institutions reported 
enrollments for each group for the entering cohorts. 

Across all groups except Asians, DePaul’s graduation rates are at least 7 points higher than the 
median of the reference institutions. For most groups DePaul ranked between the 50 lh and 75 th 
percentiles of the reference group distributions meaning that in most cases, DePaul’s graduation rate is 
at or above what we should expect given the reference point. For the entire 1993 cohort DePaul’s 
graduation rate (58.1%) ranked at the 67 th percentile of the distribution compared to the reference group 
median (44.4%). Like the reference group, graduation rates for DePaul females (59.6%) are slightly 
higher than those of males (56.0%). The difference between males and females in the reference group 
(3.6 points) is identical to the difference between rates for DePaul males and females. For both males 
and females, DePaul’s retention rate exceeds that of the reference group median by 14 percentage 
points. However, in terms of the distribution, DePaul males ranked at the 70 th percentile while females 
ranked between the 65 th and 66 th percentiles. Therefore, in terms of expectations based on the reference 
group median, the retention rates for males at DePaul are better than females. 
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Figure 6b 

DePaul Graduation and Reference Group Graduation: 1993 Entering Cohort 

X DePaul Graduation Rate (Source: OIPR Retention Database) 

— Reference Group Distribution Median (Source: CSRDE) 
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Sixth-year graduation rates for African-Americans are lower than those of other ethnic groups, both for 
DePaul (48.8%) and the reference group (Median -30. 8%). The range of graduation rates for African- 
Americans in the reference group was the largest of any group, ranging from 8.4% to 66.0% across 
seventeen institutions. In terms of the reference group distribution DePaul is performing better than 
three-quarters of its peers in terms of African-American graduation rates. So while graduation rates for 
African-American students are lower than students of other ethnicities, this is not an uncommon 
phenomenon. Hispanic students in the reference group had a median graduation rate of 47.1%, about 8 
points below DePaul, which falls in the 64 th percentile of the distribution. There was also a great deal of 
variation in the reference group for Hispanic graduation rates, which ranged from 8.5% to 64.3% across 
sixteen institutions. The only group that DePaul ranked below the reference group median (32 nd 
percentile) for graduation rates was for Asian students. Graduation rates for Asian students at DePaul 
(50.7%) and in the reference group (Median=57.2%) are the highest of any group. However, relative to 
the reference group, DePaul’s graduation rate for Asian students is below expectations. Caucasian 
students at DePaul have a sixth-year graduation rate of 61.1%, 16 percentage points higher than the 
reference group median (44.7%), and ranked at the 65 th percentile of the reference group distribution. 

Though DePaul has sixth-year graduation rates that exceed the reference group median for most 
groups, only the African-American cohort reached the 75 th percentile of the reference group distribution. 
For all gender and ethnic groups in this analysis, DePaul’s first-year retention ranked between the 32 nd 
(Asians) and the 77 th (African-Americans) percentiles for each distribution. 
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Entry/Intent Comparisons 

The studies mentioned have all involved first-time freshmen, yet these students are in the minority at our 
institution, as they are across the country. A recent investigation of baccalaureate recipients each June 
has shown an ever-increasing proportion of students who did not begin their academic careers at DePaul. 
Therefore, we have undertaken retention studies of other student populations, including transfer 
students, part-time adult students, and graduate/professional students. The studies for each of these 
different student populations provide a more complete institutional picture of student retention than can 
be understood from strictly looking at the traditional student. 

Transfer Students 

All of our studies of retention on freshmen have shown that, typically, DePaul loses about 20% of all 
new freshmen after the first year. Our sixth year graduation rate tells us that approximately 60% of the 
original cohorts of new freshmen graduate after six years. Therefore, between the second and sixth year, 
DePaul has lost another 20% of the original group of new freshmen on top of the 20% lost after the first 
year. In other words, by the end of the sixth year, DePaul has lost 400 of every 1 000 freshmen enrolled. 
For transfer students, the more 
hours of college credit a 
student transfers into DePaul, 
the more likely it is that he/she 
will complete a degree at 
DePaul. Approximately 90% 
of students who enter DePaul 
with junior or senior standing 
persist to the second year and 
over 75% receive their degree. 

Sophomore-level transfers 
have retention patterns very 
similar to those seen for new 
freshmen. Freshmen transfer 
students do not tend to fair 
well at DePaul, with over 50% 
becoming inactive by the 
fourth year - many of whom 
were not passing at the time 
they withdrew. 

Our analyses also included a study of feeder institutions, both individually and by type. We 
found that, while there was not a wide disparity in first-year retention rates across transfer institution 
types, the likelihood of a student graduating with a DePaul degree after several years of attendance 
seems to vary depending upon the type of institution from which he/she transferred. Students who 
transfer from various four-year private institutions (both in- and out-of-state) or Chicago's City Colleges 
appear less likely to complete their degree. We were also able to investigate the phenomenon of 
“transfer shock” - the decline in GPA experienced by transfer students during their first year after 
transfer (Al-Sunbul, 1987; Cejda, 1994; Hills, 1965). Typically, we saw students from two-year schools 
experiencing a slight to moderate decline in GPA (.1 to .5 points) on average. Students from four-year 
institutions experienced little transfer shock, as defined here. Because of these studies, DePaul has 
undertaken many initiatives aimed at improving transfer student success, including an improved transfer 
student orientation and specific courses targeted on new transfer students. 




Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 



- ♦- New Freshmen — O— Freshman Transfers 

Sophomore Transfer* O Upper Level Transfers 



O 

ERIC 



17 



16 



Adult Students 

DePaul University has seen an expansion of its facilities from beyond the two main campuses in the City 
of Chicago to several satellite campuses in the suburbs. These campuses are designed to cater to the 
needs of the working adult student. Annually increasing enrollments at these locations testifies to their 
success. We have undertaken several studies of adult student retention and mobility across these 
campuses to investigate how these students complete their degree programs. 

We started by creating “cohorts” of new part-time (undergraduate and graduate) students and 
tracking their course enrollments. Very quickly, we realized that looking at an annual snapshot of 
enrollment was inadequate given the transitory enrollment patterns of these students. Thus, we looked 
at credit-hour enrollments by location on a term-by-term basis. Furthermore, given the difference in 
requirements across programs, we separated out cohorts by college, level and suburban campus in the 
first term. While this was done 
for many colleges, the data 
presented here in Figure 8 are 
for the new students in the 
graduate school for business 
(GSB) who, during their first 
term, entered the University 
through one of the suburban 
locations. 

By following the 
enrollment patterns of these 
students, we found that by the 
beginning of the third year, 
half of the credit hours being 
generated by this group are 
through the main Chicago 
campus. It would appear that 
these campuses do serve as 
portals to the larger university. 

The persistence patterns lend 
credence to the transitory nature of these students’ enrollments. While there was a general decline 
within the course of the academic year from one term to the next, the beginning of a new year brought a 
slight upswing in enrollments from the previous spring term. Suburban students did not differ 
significantly in persistence to graduation from students who started at the main campus. 

Lessons Learned about making an impact 

The framework for considering the value of the various analyses in the context of the institution 
comes from the Chickering and Gamson (1987) study of the seven principles of good practice in 
undergraduate learning revised to reflect the institutional researcher/audience relationship. We would 
like our data and information to result in our audiences learning new values and shifting opinions. Given 
our various audiences, how well do we think the comparative studies we performed met the 
requirements of “Good Practice”? 

1. Good practice encourages contact between the researcher and the audience: The use of 

comparative studies provides multiple contacts with the audience. This part of the process worked well. 
Unfortunately for us as an urban institution with a high percentage of non-traditional students, we have 
had limited contact with the audience of individuals concerned about the success of these students. We 
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have worked with the various deans, but in general the comparative analyses do not automatically cause 
frequent interactions with the appropriate college representatives and the researcher. We have 
undertaken a number of initiatives to increase such contact. For example, with regards to transfer 
students, plans exist to bring together the representatives to meet with their compatriots at various two- 
year feeder institutions. In doing so, we will be able to investigate the most appropriate indices to use 
for transfers and develop a more comprehensive research agenda. Also, while we have an official center 
for advising and some functions in student personnel related to student success, this audience is only 
beginning to emerge as an organized group. The discussion of success by entry type did stimulate some 
interest with this group. 

2. Good practice develops reciprocity and cooperation among the audience. With our institution in a 
growth mode, creating a sense among some that we are currently over capacity, there has not been a 
compelling force towards improving retention. In addition, the results from the comparison of our results 
with those of the institution-level reference group did not create a sense of urgency. The differentiation 
did cause some discussion among the various audiences as to how the success of some groups - such as 
the bridge group - might be improved. The demonstrated differences in the retention of some groups, 
such as the increase among those in the 2.00-2.49 GPA group and the lower then expected retention rate 
of Asian students, prompted some discussion as to the root causes of those differences. However, the 
deans were not inclined to create initiatives that crossed college lines. One of the things we are working 
on is to create a type of “meta-audience” for retention and graduation studies built around the core of the 
advising movement. Unfortunately this movement has been slowed because several key individuals are 
involved in the implementation of the PeopleSoft ERP. 

3. Good practice uses active learning techniques. The studies resulted in involvement of the various 
audiences in both a direct and an indirect manner. The initial presentation of the results of these studies 
did stimulate some suggestions for further analyses of the data, particularly from our deans who were 
interested in their particular students. These additional analyses were completed, but there has been no 
systematic follow-up. The method of providing the audience with their own data analysis tool would 
seem to be a much better way to achieve more active engagement. One of our colleagues has built an 
OLAP cube that we are now starting to share with concerned administrators and faculty that enables the 
individual to conduct many independent investigations of student success. We are also looking at some 
off-the-shelf reporting tools that will support independent studies. The comparative study will promote 
active learning if the questions that are raised are nurtured, and if those who raise the question are 
actively involved in the follow-up research projects. 

4. Good practice gives good feedback. Many of these studies came at the request of several of the 
senior administrators. Because we had been participating in the C-IDEA exchange we had both internal 
databases that we could employ and also access to external data. Unfortunately these data and hence the 
studies were again limited to first-time full-time freshmen. It is much more complicated to give timely 
feedback on transfers students, for example. Also our conversion to a new software system threatens the 
continuity of our data and required the siphoning off of resources that otherwise would have been 
available. The use of the trends over time for first year retention seemed to be one of the better balances 
of looking at persistence and doing so in a timely manner. There is also a First-Year Program at our 
university that is more of a natural audience for the first year retention rates. Looking at six-year 
graduation rates did not seem to do more than provide some “bragging rights”. One of the other aspects 
of good feedback is the comprehensiveness of the studies. In our case we looked at persistence both for 
those at a given point and longitudinally. This did give a more complete and context-oriented set of 
feedback. 
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5. Good practice emphasizes time-on-task. The comparative study in and of itself is a presentation of a 
set of facts. The amount of time on task by the audience depends on the level of concern that exists, or 
that the research creates. These studies were moderately useful. One thing they do is that they allow for 
the creation of multiple events. We spaced them over several years to continue to keep the issue on the 
radar screen of the audience. The speed with which they can be updated is also helpful in that the trend 
lines can be updated on an annual basis. Also there is some ability to do detailed statistics at the program 
level for key indicators with an external reference and a trend line and a comparison of which internal 
group performs better and poorer. 

6. Good practice communicates high expectations. Through the use of the reference group, we were 
able to set an expectation of what our retention and graduation should be, but the fact that DePaul is 
already performing at, or above, the level of the reference group tempers some of the movement towards 
change. If it had been otherwise the results would have been much more compelling. We resisted the 
temptation to select a group of small private schools or reconstitute the reference group to make our 
institution look worse. There is some discussion about selecting a true benchmark group, particularly on 
a college-by-college basis. At present most of the expectation of improving retention comes from the 
fact that our mission says that it is the right thing to do. The operational definition of the desired 
improvement has not yet matured into focused activity. 

7. Good practice respects diverse talents and ways of learning. The use of comparative analyses is 
most compelling to those who are looking for limited key indicators and the tracking them over time, 
institutions, and internal groups. This group is well served by our comparative analyses. Those who are 

context people and people people are less well served. These studies also would lend themselves to 
estimating losses in tuition as suggested by Noel et al. (1999) but we have not formally done that yet. 
There has been some informal estimation done and it keeps the attention of financial administrators as 
one of their diverse ways of learning. 

Summary 

In summary, the main strengths of the comparative analyses are that they are simple, can be done on a 
repeatedly, can use groups such as colleges that have organizational implications, and can be “redone” 
fairly rapidly when someone has a suggestion. These studies function as “trip-wires” that can be used to 
test the sense of reality that individuals have. As such they can force attention on the relative retention 
of the students when an external group is identified. Their limitation is that they do not help understand 
the causes of attrition, and they have limitations of group size for some of the demographic breakdowns. 
They can be a cause for limited activity since they can show that the institution is doing “as well as 
others like us are doing . One final limitation is that only a very limited number of key measures can 
be considered in any given report. If multiple groups are used with more than two measures, the 
complexity increases rather rapidly. 

One of our next steps will be to strengthen our databases, distribute definitions through a data 
dictionary, and move toward the OLAP cube. In addition, we will be working to form the “meta- 
customer” for our retention studies. We have augmented our research practices based on the feedback 
from several focus groups and have started discussions on a methodology for transfer students. Also we 
are looking for best ways to provide persistence data for program review and outcomes assessment. This 
involves not only the statistics to provide but also the method for providing them from our databases. 

We are also conducting more focus groups and looking at models of individual retention to increase the 
diversity of the information we provide. 
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